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ABSTRACT 



This paper describes an alternative delivery approach for 
training personnel in observation skills using interactive multimedia 
technology. The "Classroom Behavior Record (CBR) Observation Training 
Program, " includes three videodiscs that contain video and audio scenes of 
children in classrooms /playgroups and instructional narrations. The 
seven-module computer program delivers instruction through text, graphics, 
animation, and videodisc material; provides feedback on practice activities; 
and controls delivery speeds. The CBR program is based on a well-established 
observation system used to assist in diagnosis and treatment monitoring in 
hospital and school settings. The overall design of the training program is 
based on the "Stages of Learning Model" as described by R. Gagne and adapted 
to computer-based instruction,- the CBR modules have features that support 
acquisition, fluency-building, and generalization of skills. The CBR training 
program, utilizing a version of the behavior codes revised for school 
professionals, was implemented with graduate students (n=40) enrolled in 
special education courses. Data were collected over a 3 -year time period. 

Data were gathered on usage time, learning patterns, knowledge of behavior 
codes, and speed and accuracy in coding. Training outcomes indicated that the 
program is an effective and efficient training procedure. Instructional 
design, authenticity of materials, and ability to provide learner-controlled 
instruction are important ingredients in the effectiveness of the program. 
(Contains 16 references.) (AEF) 



00 

m 

cs 

a 



Training in Observation Skills for Health Care Professionals: 

Interactive Multimedia 



Gail E. Fitzgerald 

School of Information Science and Learning Technologies 
University of Missouri-Columbia 
351 Townsend Hall 
Columbia, MO 65211 
573/882-0566 

spedfitz@showme.missouri.edu 



Polly J. Nichols 

Department of Educational Services 
The University of Iowa Hospitals and Clinics 
1716 JPP, UIHC 
Iowa City, IA 52242 
319/353-6800 
polly-nichols@uiowa.edu 



Louis P. Semrau 
Department of Special Education 
Arkansas State University 
P.O. Box 1450 
State University, AR 72467 
870/972-3061 
lsemrau@kiowa.astate.edu 



rJ 

o 

cr- 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 

EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

□ This document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 






Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or Dolicv. 



O 



me 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 

AA » ^<VV0KSoirs 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



497 



Systematic behavior observation is one method of measuring children’s behavior in ecological environments 
and studying how their behavior compares to that of other children in the same situation or how it changes 
subsequent to intervention (Shores, Jack, Gunter, Ellis, DeBriere, Sc Wehby, 1993; Walker Sc Hops, 1976). The 
main strength of a systematic behavior observation procedure is the objectivity it brings to the description of 
children’s behavior. Whether observing in clinical settings, classrooms, or playgroup situations, service providers in 
psychiatric and pediatric services find behavioral observation to be a critical skill for gathering reliable and objective 
data to aid in treatment decisions (Loney, 1980; Walker, Colvin, Sc Ramsey, 1995; Wehby, Symons, & Shores, 
1995). 



To become a competent observer, one must learn a behavior code and procedures, practice in simulated and real 
settings until proficient, meet established standards for reliability, and learn to interpret observation data. These skills 
take an investment in training and practice time. Typically, observers are trained through a combination of didactic 
instruction, work with videotapes, guided practice with a ’’master” coder, and H in situ" reliability checks. This 
training protocol is time and person intensive; it can take 40*60 hours to bring a novice observer to the minimal 
acceptable level of 80% accuracy. It is not feasible to spend this amount of training time on one assessment 
procedure within most health care training programs. As a result, full training is rarely provided health care 
personnel, and observational data are based on subjective, informal procedures. 

An alternative delivery approach for training personnel in observation skills is interactive multimedia 
technology. In the Classroom Behavior Record (CBR) Observation Training Program , video and audio scenes of 
children in classrooms/playgroups and instructional narrations are stored on three videodiscs. The seven-module 
computer program delivers instruction through text, graphics, animation, and videodisc material; provides feedback 
on practice activities; and controls delivery speeds. The CBR instructional program is based on a well-established 
observation system used to assist in diagnosis and treatment monitoring in hospital and school settings. 

Description of the Classroom Behavior Record Observation Procedure 

CBR was developed at The University of Iowa Hospitals and Clinics by the educational staff in the Child 
Psychiatry Service unit (Nichols, Robinson, Sc Fitzgerald, 1979). Its purpose is to provide a method of systematic 
observation to document children's behaviors in naturalistic settings for use in diagnosis and treatment. The original 
CBR code was comprised of 38 positive and negative behavior codes to provide a detailed and specific description of 
behavior. The coding procedure involves coding the behavior of a target child in six-second intervals, while coding 
the behavior of peers in the same situations on alternating six-second intervals. Thus, the data provide a comparison 
of the target child to his/her peers under the same situational expectations. Results provide a clinical method to 
evaluate treatment effects through the use of normative peer data (Walker & Hops, 1976). CBR has been used 
continuously since 1979 in the Child Psychiatry Service unit. Primarily funded by Chapter I funds to the inpatient 
school program, trained professional observers gather pre-and post-data for children referred to the treatment center. 
These data are used to understand the youngster’s difficulties as they occur in natural settings, to establish treatment 
goals, and to monitor treatment effects, particularly of medication. Training of professionals typically took 40-60 
hours of one-on-one training with periodic re-calibration training and checking of observer reliability. 

When the project developers sought to implement training in CBR for behavior specialists in educational 
settings, the original CBR code was found to be too complex for mastery and maintenance by school professionals. 
Through factor analysis procedures using hundreds of clinical CBR records, the categories were collapsed into 15 
behavioral codes with 4 open variables for tracking unusual behaviors as defined by the observer (Fitzgerald, Nichols, 
Sc Whittaker, 1992). The CBR-School Version Code is now implemented in various school districts and provides the 
content for the current interactive multimedia training program (Fitzgerald, Semrau, Nichols, Sc Nichols, 1997). The 
original coding procedures continue to be implemented with the new code. Target and peers are coded in six-second 
intervals on alternating intervals. Data records are used in the special education identification procedure and by 
behavioral specialists to monitor the effectiveness of educational interventions for children. The code has been 
modified by professionals who choose to look more closely at the behaviors of special groups of youngsters, such as 
early childhood groups or youngsters with autism. 
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The codes in the CBR-School Version are: 



Positive Behavior Codes 



Negative Behavior Codes 



AT 


Attend/ On-task 


IM 


Incidental Motor 


II 


Instructional Interaction 


PP 


Positive with Peer 


PT 


Positive with Teacher 


CO 


Comply 


AG 


Approval Received 


V1-V2 


Open Positive Variables 



FA 


Fail to Attend/ Off- task 


PL 


Play with Object 


MN 


Motor/ Noise Obtrusive 


DD 


Disruptive, Destructive 


NP 


Negative with Peer 


NT 


Negative with Teacher 


FC 


Fail to Comply 


DG 


Disapproval Gained 


V3-V4 


Open Negative Variables 



Making and Re-purposing the Videodisc 



The initial videodisc was created by a production team at The University of Iowa Hospitals and Clinics with 
combined funding and support from the Iowa State Department of Public Instruction. The University of Iowa WEEG 
Computing Center, The University of Iowa Audio-Visual Department, and The University of Iowa Hospitals and Clinics 
(Nichols, Huntley, Lindamann, Schiele, Johnson, & Fitzgerald, 1 985). Most of the scenes which were taped for use on 
the videodisc were authentic. Youngsters were taped in classroom and playgroup situations in a child psychiatric 
inpatient center over a period of months to gain as wide a variety of children as possible. To fully capture good teaching 
samples of behaviors, six child actors were recruited to demonstrate some of the critical behaviors needing to be taught 
for CBR. 



When the CBR Code was revised to the CBR-School Version Code, the entire videodisc was re-purposed to 
teach this new code. Additional classroom video material was secured from video archives produced through a special 
project grant from the U.S. Department of Education (Semrau & Fitzgerald, 1992). Additional practice scenes were 
chosen from these archives and an additional videodisc was made to increase the number of practice-for-generalization 
opportunities. The current training program utilizes one side of a videodisc for the acquisition and fluency-building 
modules and two additional sides of videodiscs for the generalization practice modules and reliability testing. 

Instructional Design of the CBR Training Program 

The overall design of the training program is based on the Stages of Learning Model as initially described by 
Gagne (1974) and adapted to computer-based instruction (CBI) by Criswell, 1 989. Learning is viewed as a systematic, 
developmental process with the learner progressing through stages of acquisition, fluency, generalization, and 
maintenance. Anderson (1980) describes three stages in skill acquisition: 1) a cognitive stage to learn about the skill, 
2) an associative stage in which a learner practices the skill, and 3) an autonomous stage in which a learner improves 
the skill performance. 

These models are very helpful for determining an effective instructional design for skill-based instruction. 
Criswell ( 1 989) posits that effective instruction requires a match between CB I design and learner objectives as reflected 
in progressive, changing performance. Effective instructional design should facilitate the mastery of learning objectives 
at each stage in the hierarchy. The teaching approach at each stage must change to support differing goals for the 
learning. The schema displayed in Figure 1 depicts the organization of the CBR training program. Design of the 
modules within the CBR program have differing design features to support acquisition, fluency-building, and 
generalization of skills in behavioral observation. 

Despite an overall high level of organization and structure, the modules in the training program can be 
accessed in a nonlinear manner to meet different learner preferences. Movement across learning activities is easily 
executed by opening different modules. To facilitate learner control within the modules, the program frequently stops 
and allows the learner to repeat segments, advance or restart drills, and check scores. In the tutorial modules, all 
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subjects are listed on a pull-down menu which can be accessed at any time. Advisement for the learner is inserted 
into feedback on the drills which suggest levels of proficiency for each fluency drill. In early studies with this 
program, differences were observed in users' sequences, although most users follow the organizational pattern and 
embedded structure within the training program (Fitzgerald, 1995). 



Classroom Behavior Record 




Routines 

Figure 1. Matching CBR Training Modules to the Stages of Learning Model 



Acquisition 

The learner's instructional goal at the acquisition level is to acquire new knowledge or skills. The CBR 
training program includes a series of tutorials with short practices embedded within each concept or skill. The 
teaching hierarchy is simple to complex, based on the structure of the knowledge and skills needed in this 
observation system (Kelly, Orgel, & Baer, 1985). These procedural tutorials provide information through narration, 
screen notes, video demonstrations, and guided practice segments with feedback. 

In the Observation Procedures module, tutorials are used to explain the importance of gathering objective 
observation data, studying behavior within contexts, using the open variables to define behaviors of special interest, 
following a hierarchy of rules for making coding decisions, and coding the child and peers' behaviors on alternating 
intervals. A four-minute practice is provided for the user to apply the coding rules of precedence. The practice stops 
at the end of each minute to allow the user to enter his/her codes and compare entries to correct answers. Another 
practice activity allows the user to practice the alternating interval routine with a group of four youngsters. The 
computer guides the user through coding one student as the target and the other three peers using the alternating 
interval routine. The practice stops each minute to allow the user to enter codes, see answers, and repeat if desired. 
See Figure 2 for an example of a tutorial screen in the Observation Procedures module. 

In the CBR Codes module, the behavioral codes are taught in four groups of related behaviors: task-related 
behaviors, interactive behaviors, compliance, and student as object. For each code, the behavior is defined and three 
video examples are shown. In addition, brief practices are available within each category for practice differentiating 
and using the codes. When errors are made, feedback is provided to explain the critical attributes of each code. These 
practices are untimed since they focus on mastery of the codes, not speed of coding. Figure 3 displays the tutorial and 
practice choices available for the group of task-related behaviors. 
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Fluency 



Figure 2. Tutorial Screen for the Observation Routine Practice 



Figure 3. Tutorial Screen for Learning Task-Related Behavior Codes 




Following acquisition, the goal of instruction is for the learner to use the knowledge and skills in a fluent 
manner (Criswell, 1989). New information should become automatic so it can be recalled without undue cognitive 
processing. The CBR training program includes two components for building fluency. 

In the Text Flashcard module, brief descriptions of behavior are displayed in text on the screen. The format for 
the text flashcard is displayed in Figure 4. There are approximately 100 text flashcards which are presented in random 
order at three controlled speeds, allowing a limited time to read the description and enter the user's code into the 
computer keyboard. The correct code is given and explained if the user enters an incorrect code or if time runs out 
before the code is entered. A running score is provided the user along with suggested levels of achievement for 
increasing the practice speed. The goal is to obtain fluency at the six-second speed needed for actual behavior coding 
with the CBR procedure. Rapid responding to these flashcards builds automaticity in remembering the codes. 
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Figure 4. Text Flashcard Drill Card 

The second fluency-building module, Video Flashcards, provides a more realistic coding simulation because it 
uses actual video scenes of children. In addition to continuing to build speed and accuracy, the fidelity of the practice 
is increased in this phase of instruction (Alessi, 1983). The format for the video flashcard is displayed in Figure 5. 
The flashcard module has 100 brief video clips displaying classroom behaviors which are displayed in random order. 
The user is allowed 3 seconds to enter his/her code following the video segment. The correct code is given and 
explained if the user enters an incorrect code or if time runs out before the code is entered. A running score is 
provided the user along with a suggested level of achievement to have mastered the fluency module of the training. 
Through these video flashcards, users increase their ability to rapidly interpret and code behaviors youngsters’ 
behaviors in a wide variety of situations. 




Figure 5. Video Flashcard Drill Card 
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Generalization and Maintenance 



Goals at the generalization stage are for the learner to apply knowledge or skills in the way they will be used 
"in situ'' and to practice to mastery. The practice situations must be similar to those used during instruction but offer 
mixed practice with new, realistic scenarios (Kelly et al., 1985). Classroom and playgroup scenes are included to 
provide extensive practice for the learner which simulates the job of an observer in psychiatric and educational 
settings. 

There are a total of eight scenes in the Group Practices module providing practice opportunities with a total of 
18 different youngsters varying from ages 5 to 15. The scenes show youngsters in play group and classroom 
situations. These practices follow the actual CBR procedures: the user alternately codes the target student and peers 
with a continuous six-second interval and marks his/her codes on a paper/pencil recording form. Each segment runs 
for 10 intervals (1 minute) and then pauses to allow the user to enter his/her codes and compare codes to those of the 
master coder to discern differences and compute reliability based on the 10 intervals. The user may choose to repeat 
each minute for review or continue forward. Figure 6 displays the format for the data entry and comparison. In this 
example, Mark is the target child and BJ and Ky are peers. 




Figure 6. Data Entry Screen for Classroom Application 

The CBR program includes two methods for testing mastery of coding and reliability. In the Codes Test 
module, 30 video flashcards are displayed for six-seconds each and the user marks the code on a paper/pencil recording 
sheet. This test is designed to test accuracy in coding and allows the user adequate time to think and enter his/her 
codes into the computer. Approximately half the video segments are new to the user as they were withheld from 
prior practice. Results are provided the user following the test. 

The second testing module, Reliability Tests , operates the same as the Group Practices. Four youngsters are 
displayed in the video, and each may be selected as the target child; thus, four reliability tests can be run in this 
module to check mastery and maintenance over time. The test is five minutes long (50 intervals) and the user codes 
target and peers on alternating intervals, marking the codes on a paper/pencil coding sheet. When the test is 
completed, the user enters his/her codes into the computer program and receives a reliability score. 
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Methodology and Outcomes 



Subjects and Training Procedure 

The CBR training program has been implemented with 40 graduate students enrolled in special education 
courses at a mid-Atlantic university. These students were enrolled either in teacher certification programs in learning 
and behavioral disorders or in the doctoral program in special education. Data were collected over a three-year time 
period utilizing multiple instructors. 

Following a general demonstration of the CBR training materials and overview of the coding categories, 25 of 
the users proceeded independently through all modules of the training program. For 15 of the users, a course 
instructor utilized the acquisition modules with the class as a whole prior to their independent use of the program. 
Proficiency tests were scheduled approximately 5-6 weeks after independent practice started. If required reliability 
levels were not achieved at the initial test session, additional practice and re-testing were required until reliability 
standards were met. 

The data reported below are the outcome scores for each user on his/her first attempt on the outcome tests. 
Users were not certified for using the code and procedure in actual educational or clinical practice until the 
certification criteria were met. Certification requirements were met by one of two score combinations: 72% on codes 
test and 80% on reliability test, or 80% on codes test and 78% on reliability test. With extended practice, ail but two 
users met criteria to be certified in CBR for actual use with children. 

Training Outcomes 

Usage Time. The average length of program engagement time for users to pass reliability testing equalled 
15.68 hours. The range extended from 5.18 hours to 36 hours of practice time. These data do not include off-line 
preparation time students may have spent studying the codes. 

Learning Patterns. Averaging across all 40 users, approximately 30% of on-line training time was spent in 
the acquisition stage of learning the code; 25% of on-line training time was spent in the fluency-building stage of 
learning; and 45% of on-line training time was spent in the generalization stage with simulated practice. 

Outcomes on the Codes Test. The Codes Test measures knowledge of the codes. The mean score for the 
users equalled 76.9% (s.d. = 8.875) on their first attempt on the test. 

Outcomes on the Reliability Test. The Reliability Test measures both speed and accuracy in coding. The 
mean score for the users equalled 80.125% (s.d. = 6.086) on their first attempt on the test. 

Summary 

The Classroom Behavior Record Observation Training Program appears to be an effective and efficient 
training procedure for such a comprehensive code and complex observation procedure. Comparing the training time 
to records established at The University of Iowa Hospitals and Clinics with the original 38-item code, training time 
has been reduced from 40-60 hours to 16 hours. Some of this training time difference may be related to the reduced 
complexity of the CBR-School Version Code with 16 categories in comparison to the 38 categories in CBR. 
However, it is noteworthy that these training results were achieved with minimal direct training from an instructor or 
a master coder sitting side-by-side with a novice observer for extensive practice using videotapes or live classrooms. 

The instructional design of the training program, the authenticity of the materials, and the program's ability to 
provide learner-controlled instruction are important ingredients in the effectiveness of this interactive, multimedia 
training program. While the stages of learning model provided the overall structure for designing the training 
activities, its learner-centered, hypermedia interface allows users to select their own paths through the stages of 
acquisition, fluency, and generalization and move from cognition to autonomous skill performance. 
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Dissemination 



As provided by grant #H029K30210 from the U.S. Department of Education grant, copies of the CBR 
training program are being distributed to training institutions, state departments of education, and inservice agencies. 
The materials are available for both the Macintosh and Windows platforms. Needed equipment, in addition to a 
computer system, includes a Pioneer, Panasonic, or Sony videodisc player with monitor and connecting cable. The 
only cost for the materials is a postage and handling charge. Eligible agencies must file an implementation plan for 
training initiatives. Complete information on the programs is available by contacting the authors or through the 
project website at: http://tiger.coe.missouri.edu/tpss.html. 
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